Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3rd party broker #5015

Open
wants to merge 23 commits into
base: main
Choose a base branch
from
Open

3rd party broker #5015

wants to merge 23 commits into from

Conversation

hardillb
Copy link
Contributor

@hardillb hardillb commented Jan 15, 2025

part of #4969

Description

Adds APIs for provisioning 3rd Party MQTT Broker credentials for a team

  • /api/v1/teams/:teamId/brokers
    • GET - returns list of 3rd party broker credentials (sanitized of 'sensitive' credential info)
    • POST - creates a new 3rd party broker credential object
  • /api/v1/teams/:teamId/brokers/:brokerId
    • GET/PUT/DELETE - operations on individual broker creds object
  • /api/v1/teams/:teamId/brokers/:brokerId/credentials
    • GET (only accessible via agent token to retrieve credentials for use)

:brokerId is the usual hashid of the broker object's id. However, we will use team as a special case brokerId when refering to the team broker - which will have some particular behaviour. Details on that still to be figured out.

Related Issue(s)

#4969

Checklist

  • I have read the contribution guidelines
  • Suitable unit/system level tests have been added and they pass
  • Documentation has been updated
    • Upgrade instructions
    • Configuration details
    • Concepts
  • Changes flowforge.yml?
    • Issue/PR raised on FlowFuse/helm to update ConfigMap Template
    • Issue/PR raised on FlowFuse/CloudProject to update values for Staging/Production

Labels

  • Includes a DB migration? -> add the area:migration label

Copy link

codecov bot commented Jan 16, 2025

Codecov Report

Attention: Patch coverage is 68.84058% with 43 lines in your changes missing coverage. Please review.

Project coverage is 78.33%. Comparing base (4c47024) to head (1a1a647).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
forge/ee/routes/teamBroker/3rdPartyBroker.js 57.53% 31 Missing ⚠️
forge/db/views/BrokerCredentials.js 68.75% 5 Missing ⚠️
forge/routes/auth/index.js 20.00% 4 Missing ⚠️
forge/db/controllers/AccessToken.js 85.71% 1 Missing ⚠️
...igrations/20250109-01-EE-3rd-party-broker-creds.js 83.33% 1 Missing ⚠️
forge/db/models/BrokerCredentials.js 96.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5015      +/-   ##
==========================================
- Coverage   78.41%   78.33%   -0.09%     
==========================================
  Files         329      333       +4     
  Lines       15494    15632     +138     
  Branches     3579     3603      +24     
==========================================
+ Hits        12150    12245      +95     
- Misses       3344     3387      +43     
Flag Coverage Δ
backend 78.33% <68.84%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@hardillb
Copy link
Contributor Author

Team delete needs to also remove all 3rd party broker creds and shut down any mqtt-schema-agents that might be running.

container driver needs to look at DB on startup and restart any missing mqtt-schema-agents

@hardillb hardillb requested a review from knolleary January 21, 2025 16:39
@hardillb
Copy link
Contributor Author

Should be enough to start the UI work to create/edit these

@hardillb hardillb marked this pull request as ready for review January 21, 2025 16:40
@knolleary
Copy link
Member

Some api tweaks as discussed:

/api/v1/teams/:teamId/broker/credentials -> /api/v1/teams/:teamId/brokers

  • GET lists brokers
  • POST adds new broker

/api/v1/teams/:teamId/broker/:brokerId -> /api/v1/teams/:teamId/brokers/:brokerId

  • GET/PUT/DELETE - operations on individual broker creds object

/api/v1/teams/:teamId/broker/:brokerId -> /api/v1/teams/:teamId/brokers/:brokerId

  • GET/PUT/DELETE - operations on individual broker object

/api/v1/teams/:teamId/broker/:brokerId/credentials -> /api/v1/teams/:teamId/brokers/:brokerId/credentials

  • Agent only api for retrieving creds

Team Broker

/api/v1/teams/:teamId/brokers/team - team is the magic id of the built-in team broker

Topics info

/api/v1/teams/:teamId/broker/:brokerId/topics or /schema or whatever is appropriate...

@hardillb
Copy link
Contributor Author

@knolleary renaming done.

Still can't get codecov to show me what's missing, I think you now need to create an account even for the opensource/free access.

@hardillb
Copy link
Contributor Author

For the POST the body needs to looks like

{
  name: { type: 'string' },
  host: { type: 'string' },
  port: { type: 'number' }, // default 1883
  protocol: { type: 'string' }, // mqtt: or ws:
  protocolVersion: { type: 'number' }, // this should be 3, 4 or 5 picked from a list of 3.0, 3.1, 5 (with 3.1 as the default)
  ssl: { type: 'boolean' }, // default false
  verifySSL: { type: 'boolean' }, // default true
  clientId: { type: 'string' },
  credentials: {
    type: 'object' // MVP keys of strings, username and password
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants